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ABSTRACT 

This paper summarizes a review of the recent 
literature in search of evidence for the validity of industrial 
assessnent centers* The topic is divided into two parts: (1) the 
evidence regarding the validity of several of the individual 
assessment techniques that are used in industrial assessnent centers; 
and (2) the evidence concerning the validi.ty of the ovorai assessment 
rating (OAR) - Findings indicate that: (1) the most valid techniques 
seea to be performance exercises involving "samples” of behavior; (2) 
few studies have treated the technique psychonetrically ; (3) numerous 
reports fail to report the correlations of the tests, games, etc. 
with any sort of criteria, although these data were gathered; (4) the 
overwhelming finding is that OA B 1 s are related to a variety of 
criteria for subsequent performance and potential in managerial 
positions; (5) there are serious questions regarding the quality of 
research designs and problems with criterion contamination in some 
studies; and (6) the overall assessment evaluation adds a unique 
contribution to assessment beyond information obtainable from 
objective tests. (Authoc/TA) 
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This paper summarises a review of the recent literature in search 
of evidence regarding the validity of industrial assessment centers. 

The topic has been divided into two parts: (1) the evidence regarding 

the validity of several of the individual assessment techniques that 
are used in industrial assessment centers, and (2) the evidence concern- 
ing the validity of the overall assessment ratings (OAR). In addition, 
the paphr reports the lack of evidence presented with regard to the ef- 
fectiveness of assessment centers as an approach to management develop- 
ment. The author recognises that assessment centers are used for many 
purposes, but has chosen to limit his attention to the above areas for 
these seem to be the most important ones and they are the ones about 
which the most has been written* Data for the paper were gathered by 
consulting references in the standard published sources, reviewing 
transcriptions of speeches 8iven at various conferences and conventions, 
and obtaining in-house materials from companies sponsoring assessment 
centers. Letters requesting material were sent to all companies known 
to have assessnent centers. 
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Validity of Individual Assessment Techniques 



One of the outstanding features of the assessment center approach 



to personnel evaluations is the gathering of multiple assessments by 

<#■ 

multiple assessment techniques. The synthesis of the assessment data 
vO 

° O by several assessors during the review period may be built on complex 
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interactions of observations which take place over a period of tiuie. 

The examination of validity of individual assessment techniques may 
not adequately reflect the complex nature of multiple assessments. 

At the same time knowledge of the contribution of each technique is of 
practical ^nd theoretical value. 

Table 1 presents a summary of the relationships of several assess- 
ment techniques and internal and external criterion. It is important 
to note that the individual assessment techniques cover a broad range 
of devices including objective and subjective tests, individual and 
group exercises, and games and discussions* It is particularly impor- 
tant to note that many of the devices are performance exercises that 
simulate major dimensions of managers 1 jobs. The table does not reveal 
the wide variety of ingenious exercises which have been devised. Num- 
erous varieties of in-baskets, role-playing exercises, games, etc. have 
been developed. Unfortanately, little systematic dat^ has been gathered 
to evaluate their effectiveness. 

The criteria used for this first part of the paper include both 
internal criteria (e#g. ratings by assessment center staff, and overall 
assessment ratings (OAR) ) and external criteria (e*g. current rank in 
company, subsequent promotion, salary progress, and managers 1 ratings). 
Validation against the CAR assumes thet this summary is a relevant, 
worthwhile criterion. It is a composite of evaluations from multiple 
assessors often making observations from multiple techniques, but to 
assign the CAR the status of a ''criterion 11 may be premature. On the 
other hand, in many situations no batter evaluations of managerial 
characteristics exist. More helpful information can be obtained by 
examining the relationships with external critetla. These data reveal 
validity against measures independent of assessment. 
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To adequately review the validity of the individual assessment 
techniques would require references to a large body of previous literature 
in which the techniques have been used by themselves for assessment and/or 
management development* This paper deals only with the data generated 
directly from assessment centers* Few studies have treated the per- 
formance teat* psychometric ally, that is, generated quantitative data 
directly from participation in the exercises and correlated these with 
criteria. For the most part, Judgments and ratings are made after the 
assessors have observed the participants carry out the exercises. 

In-basket 

An in-basket is a managerial Job simulation in which the participant 
must deal with memos, letters, manuscripts, messager, reports, etc. that 
commonly would be found in a manager's in-basket. The manager is given 
appropriate background information about the simulated organization 
he is to work for, and the job he Js to perform. The instructiona 
usually emphasize that the manager is not to play a role or say what he 
would do with ’the problems presented to him, but rather that he is to 
perform as if he, himself, were actually on the job. Each item of the 
in-basket is a document or set of documents that deals with a particular 
administrative problem* The responses of the manager arc his work on 
the problem posed by the items*, this may take the form of written notes, 
memoranda, letters, etc. Following the managers* responses, he is 
usually asked to explain the reasons for his actions. 

Bray & Grant (1966) asked assessors to read a narrative report 
of the asoessee's performance in the in-basket itself and in the dis- 
cussion of reasons for action, and to rate overall in-basket perform- 
ance on a 5-point scale* Mean ratings were correlsted with assessment 
staff evaluations and eubpequ^nt criteria. The ici-basket contributed 
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a sizeable amount to the GAR for both college and no-college groups. 

Against an external criterion of salary progress the in-basket correlates 
significantly for 5 of 7 sub-groups, Wollowick 6. McNamara (1969) 
found a correlation of 32 between judged in-basket performance and a 
criterion change of position level over a three-year period, 

LeaderlesQ Group Discussion - 

In a leader less group discussion, a group of participants is given 
a problem to solve and instructed to arrive at a group decision 
within a specified time period. Often participants are assigned points 
of view or roles to pley. For example they may be instructed to try to 
get as much salary increase for their subordinate or budget for their 
department, but at tbe same time facilitate a harmonious decision. 

Assessors observe which participants assume various leadership roles, 
how individuals are accepted by the group, and the influence each 
participant has on the final decision, 

For the college and non-college groups in the Management Progress 
Study, Bray & Grant (1966) found that judgments of assessors observing the 
Jaaderless group discussion yielded the most consistently significant 
relationships with CAR and salary progress, Wollowick 6 McNamara 
(1969) also found a significant correlation with the longitudinal criterion. 
Management G a mes 

Management games are another form of management simulation. They 
usually require the participant to work* cooperatively and/or compet ively, 
'*ith other participants* Whereas the in-basket measures a narrower 
range of abilities centered around administrative skills, games usually 
tap a broader spectrum of skills Including ability to work under stress, 
leadership, interperconal relations, and organization and planning. 

Because there is such a wide variety of games in use, it is hard to 
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generalize about their makeup. Two games which contain some common 
features of many others are briefly described: 

Conglomerat e - T rt ams of participants ?.c ade shares of company stock 
in order to form conglomerates. To win, s company must plan and 
organize its activities during three fast-paced trading sessions. Be- 
havior flexibility, interpersonal sensitivity, operation under 
stress and leadership are some of the evaluations made by assessors. 

The Keyboard Problem - Teams of six candidates play the role of 
the Board of Directors of a small business concern. They buy and 
sell keys used on Keyboard consoles used in computer products. 

Given a selection of keys differing in profit margin each team 
must decide hov to invest its money* organize its purchasing, 
control stock and sell its product. Assessors watch for indications 
of emergent leadership, organizational ability, financial acumen, 
quickness of thinking and operation under stress. Adaptability 
is observed x>dien bonus options occur throughout the problem 
calling for the drastic redeployment of resources. 

Hardesty & Jones (1968) found no relationship between .performance in a 
stock market game and the OAR but a manufacturing problem was significantly 
correlated with both CAR and subsequent salary progress in the AT^T 
studies (Bray 6 Grant, 1966). The game seemed somewhat more predictive 
for the non-college samples than for the college samples. A manu- 
facturing game also showed predictive validity at I&M (Vollowick & 

McNamara, 1969). 

Paper and pencil teste 

As one might expect, a vide variety of tests have been used in 
assessment centers. In no case have tests been used In a strictly 
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psychometric manner by setting up cutting scores and the like. Usually 
the tests are considered during the review period, along with all other 
data* to form the overall evaluation of the participant* 

In the cases where we have data on the role uhat intellectual tests 
play in assessment center results, the findings are generally mixed. 
Since a wide variety of tests have been used with a number of different 
managerial positions and since cross-validation is seldom possible, 
it is hard to generalize about the finding or make firm statements 
about the applicability of specific tests* On thf other hand, the 
results are such to warrant the recommendation to include some test 
of intellectual ability in the assessment center battery of assessment 
techniques. 

Bray & Grant (1966) used three mental abilities test*; in the 
Management Progress Study: School and College Ability Test (SCAT), 

Critical Thinking in the Social Science Test, and the Contemporary 
Affairs Test. All three of the tests correlated significantly with the 
staff prediction » »f success and with t tie otaff rating on General 
Effectiveness. Correlations were found to bs higher for the non* college 
groups, but in all cases there was much variance in the Judgments not 
accounted for by the intellectual tests. Since the Verbal portion of 
the SCAT correlated highest with the staff judgments it was included in 
further analyses that studied the relative contributions of several 
assessment techniques. In the college sample the SCAT was least 
Important (following the three situational exercises), but in the non- 
college sample it was second most important* When the three tests were 
correlated with the criterion of salary progress, the results are highly 
variable, but the SCAT Verbal showed a significant relationship 
for four of the seven groups* In one of those cases, though, the 
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correlation was negative • Bray & Grant (1966) give no explanation 
for this unusual finding. While we can see that the mental ability 
tests do contribute to the predictions in the Management Progress Study, 
it Is clear that they do not tell the whole story. When the mental 
ability tests scores are partialed out of the staff judgment ratings, 
these staff Judgments still correlate significantly with salary progress. 
This indicates that the more elaborate assessment process makes additional 
contributions over what is possible with only the mental tests. For the 
Management Progress Study, then, Bray & Grant (1966) conclude that, 
even though situational tests contri >ute more to the assessment paper- 
and-pencil instruments measuring mental ability should be included in 
the battery of techniques. 

Similar recults have been found in a number of other studies. 

Hardesty & Jones (1968) found correlations over .40 between the parts 
of the School and College Ability Test and the Miller Analogies Test 
and the Potential Rating given by the entire assessment staff. The 
Potential Rating was a Joint decision of two professional assessors 
acid three managers as to the individual's probable level of attainment 
in the company. While these correlations are significant and there are 
significant differences in the average test performance between high 
potential and not high potential individuals, the authors present 
graphs which show that there is great overlap of the distributions of 
these two groups* In other words, many individuals rated high potential 
get scores on the test lower than those rated not high potential. 

Bray 6 Campbell (1968) report on the use of an assessment center to 
select salesmen in which the SCAT, Critical Thinking, and Abstract 
Reasoning tests were administered. The three tests were correlated 
with a set of field ratings of sales performance .25, .26, and .02 
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respectively. 

Tests of intellectual ability have not universally proven valid 
in assessment center settings, Intelligence tests were used in the 
assessment program reported by Hinrichs (1969), The Concept liastery 
Test (CMC) and the School and College Ability Test (SCAT) Numerical 
Part were correlated with external (relative salarj** standing) , internal 
(assessment staff evaluation) and parallel (managerial potential evaluation 
following review of normal personnel data) criteria. There was esentially 
no correlation between the mental ability testa and the c/iter it. in 
this study. This lack of correlation does not, of course, mean that 
there is no relationship between mental ability and success, and it 
should be noted that the study is concurrent iii design. Tha more 
important information would be evidence regarding the predictive 
validity of the tests in relation to subsequent, criteria of managerial 
success • Data on this last point has been reviewed in the discussion 
of the Bray and Grant (1966) report of the Management Progress Study, 

Other studies with a predictive validity design are those of 
Albrecht, Glaser, and Marks (1964) and Wollowick and McNamara (1969). 
Albrecht, et. al« (1964) used the Problems test (a brief mental ability 
test) and the Watson-Glaser Critical Thinking Appraisal (a tast of 
functional effectiveness in applied reasoning). The criteria were ratings 
on the following Job performance dimensions: forecasting and budgeting 

effectiveness, sales performance , effectiveness in interpersonal 
relationships, and overall effectiveness? These ratings were made 
by four different Judge groups: distric 1 managers, regional managers, 

peers, and consultants* Four of the possible 32 correlations (2 tests, 

4 criteria, 4 work areas) were significant, whereas 9 of 16 coirelations 
involving the overall assessment ratings were significant. 
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The criterion in the predictive study of Wollowick and McNamara 
(1969) was the increase in managerial responsibility after the assess- 
ment procedure and it included a consideration of number of persons 
Supervised, job complexity, and financial and skill responsibilities. 
Correlations of the SCAT-Total (.11) and the Otis (.07) were not 
significant. On the other hand, the SCAT Verbal and Quantitative 
correlated significantly with a composite criterion of behavioral 
ratings in a study at S0HI0 (Carlton, 1970). 

The experience with personality tests in assessment centers has 
been a mixture of moderate to little success. This experience reflects 
the general level of success with these tests in other managerial 
selection and placement settings. Personality tests have been used 
in almost all centers and continue to be used deBpite moderate success. 
Job analysis of the managers^ job show that certain affective and 
interpersonal characteristics exist and should prove useful in assess- 
ment. These same characteristics can also be observed in other ways 
and probably are in the many activities of en assessment center. 

In the Management Progress Study, Bray & Grant (1966) used two 
multi-scale tests: The Edwards Personal Preference Schedule (EPPS) 

and tie Guilford -liar tin Inventory of Factors GANCN (GM) . Only 2 of 
the 13 scales of EPPS and 1 of the 5 scales of U r t 1 correlated with 
the overall staff prediction. When the scores were correlated with the 
subsequent criterion of salary progress, the results were mixed for 
the seven separate groups studied » The most success was found in one 
group in which 5 of the EPPS were significant* For the other groups 
the EPPS was not predictive and in all but one group GM M 

completely failed to predict the long range criterion. 

For ttinrichs (1969) the Allport-Vernon-Lindeey Study of Va lues did 




9 



*w -< .-».-«*i«*^ip. ww--.- 



not correlate with the concurrent criteria. To a small extent, the 
Gordon Personal Preference Schedule (GPP) did correlate with the 
internal criterion and one scale of the same test also predicted 
the long range criterion in the Wollowick and McNamara (1969) study. 

Carleton (1970) Reports the predictive correlations of person- 
ality tests with both a global composite criterion (based on salary 
growth and promotions over a 4-year period) and managerial ratings 
of 13 scales reflecting important managerial characteristics, e.g. 
amount of participation, oral communication, impact, etc. The Gordon 
Survey of Interpersonal Values (SIV) and the Welsh Figure Preference 
(WFP) both showed a couple of correlations with the global criterion, 
but less ability to predict the rating scales, even those scales 
supposedly reflecting non-cogaitivo behaviors. 

Projective Techniques 

Grant, Katkovsky, and Bray (1967) report that projective techniques 
made contributions to the predictions of the assessment center staff 
and to a subsequent criterion of salary progress. After reviewing 
the results of three projective techniques, clinical psychologists 
rated the managers on nine rating scales for variables such as 
optimism- pessimism, general adjustment, self-confidence, leadership 
role, etc. These ratings were found to correlate with various com- 
ponents of the assessment center staff evaluations and the overall 
staff prediction. The variables measuring leadership and motivation 
proved more valuable than those measuring adjustment and other personality 
factors. 

Grsnt, et. aU (1967) conclude that the projective techniques 
make a significant contribution to the assessment process, but very 
carefully warn against the over-generalisat ion cf these results: 
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The conditions under which these encouraging results were obtained 
should be noted. The projective report is a summary report 
based on three different instruments. This report is in no way 
a scoring of the projective protocols sit is impressionistic in 
nature. Neither is the report deeply "clinical;" every effort 
was made to orient it to the motivations relevant to business 
management* Finally, because four-fifths of the reports were written 
by one psychologist a question could be raised regarding the 
replicability of the findings* 

Finley (1970) reported that 13 psychological predictions based on 
information from two projective techniques were correlated both with 
the predictive ratings from the entire assessment center staff after 
considering all sources of information and with supervisors' ratings 
gathered several years later* The projectives did not correlated with 
supervisors' ratings in as many cases as they did with the committee's 
ratings, but the large number of significant correlations suggest 
future research may be fruitful* 

Several coroments are warranted with regard to this study. It 
should be recognized that there was contamination of both the committee 
ratings and the supervisory ratings from knowledge of the projective 
information and the supervisora* ratings were further contaminated 
from knowledge of the committees ratings at the end of the assessment 
program* Finally, there was no cross-validation of the results, 
although replication was present it. the form of separate analyses for 
two different groups of managers. 

Intervi ew 

Contrary to much of the literature on interviewing, it has been 
found by Grant 6t Bray (1969) that the interview can make a positive 
contribution to the assessment management potential# Since this study 
is so controversial and relevant, it warrants some detailed attention. 

The interviews were unstructured and conducted by psychologists as 
a part of the Management Progress Study at AT6T (Bray, 1964). Inter- 



o 

ERIC 



11 



viewer reports were coded by judges and rated on 18 variables, including 
such things as Personal Impact, Oral Communications Skills, Behavior 
Flexibility, etc* Inter-code reliabilities were generally in the *80' s 
with only 6 of 36 correlations (10 variables for 2 samples) below *70. 
Correlations of interview variables with several composite assessment 
variables based on all assessment techniques showed that* the interview 
made substantial contribution. The most important interview variables 
seemed to be personal impact- force fu loess, oral communication skills, 
energy, and need advancement. Correlations of interview variables 
with staff predictions of whether the participant would reach middle 
management within 10 years revealed 22 of 35 significant relationships* 
Finally, 18 of 36 correlations between interview variables and a sub- 
sequent criterion of salary progress were significant. The authors 
clearly acknowledge that these analyses do not show the relative effective- 
ness of the interview in comparison with other assessment techniques. 

The study does establish* that the interview (In this case unstructured 
and carried out by trained professionals) makes a reliable and valid 
contribution to the assessment of managerial potential. 

Summary 

To summarize the information in this section, I might make the 
following points: 

(1) The most valid techniques seem to be performance exercises 

■ involving ''samples" of behavior. These are to be contrasted 
with the more traditional paper-and-pencil tesfcp which might 
be called "signs" of subsequent behavior. 

(2) Note that the interview and the projective devices make valid 
contributions in at least one stidy. This is an exception 

to the first point Just made and is in contrast to the vs<it 
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amount of generally negative literature on these cechniques 
in previous industrial psychology studies. 

(3) Few studies have treated the techniques psychometric ally. 

Rather judgments or ratings are made after the assessors 
have observed the participants carry out the exercise. 

Thus, the validity data are usually a confounding of behavior 
elicited and judgments as recorded by the assessors. 

(4, Numerous research and technical reports fail to report the 
correlations of the tests, games, and other exercises with 
any sort of criteria, even though these data were gathered. 
Practical efficiencies and theoretical advances could be made 
with very little additional expenditure of energy. 

Validity of Overall Assessment Ratings 

To address the second general area, the paper reviews the validity 
of overall assessment ratings {*AR) and recommendations resulting from 
assessment and the effectiveness of assessment centers as a management 
development technique. In this section, the OAR is considered the "in 
dependent variable" or the "predictor variable," the validity of which must 
be established by reference to concurrent and/or subsequent criteria. 

Table 2 sunmarites the validity of the OAR a3 a predictor of management 
potential. Ideally, we would want evidence derived from longitudinal, 
predictive validity designs, but ve shall see that concurrent studies 
have been carried out. The review of research will be concerned with a 
variety of criterion measures including administrative decisions such as 
salary, salary progress, end promotions, and also some more behaviorally 
oriented criteria such as observations of Job performance by rerearchers 
and supervisors. To the extent possible, we will include evidence that 
shows the validity of assessment centers over and above other approaches, 
such as tests and supervisors evaluations of potential. 

13 



We have attempted to organize the validity research along a continuum 
of quality of research design. At the one end are those designs' which 
yield the most conclusive and generalizable results. These would be long 
range longitudinal studies involving representative samples with little 
or no criterion contamination and valid criteria. At ttie other end are 
those designs which are concurrent, involve small and unrepresentative 
samples, avA involve poor criterion likely to be contaminated by knowledge 
of predictor data. 

Table 2 contains sufficient detail that the reader can get a pretty 
good understanding of these several research studies. Therefore, the body 
of the paper will not repeat a description of all of the detail of the 
research projects. The first two studies are the only two which made no 
operational use of the OAR. This avoids the possibility of criterion con- 
tamination through knowledge of the predictor data. The results show valid 
pre' let ion of the following criteria; salary progress, management level 
attained, and ratings of sales performance by a specially trained field 
review team. In the Bray & Grant (1966) study the CAR correlated moderately 
high with the criterion of salary progress in four separate validation 
groups. For the total sample expectancy data are presented. The table 
shows that a significantly greater portion of those who received an OAR 
of l, Yes, will attain third level management in 10 years” actually 
had progressed that far in comparison with those who received a less 
favorable (WR. These criteria were gathered anywhere from 5-8 years 
after the assessment and the results are even more clear-cut after a full 
10 years (Bray, 1971; personal communication). The sales recruits study 
(Bray & Campbell, 1968) showed a significant correlation of OAR and ratings 
by field review team, but no correlation against ratings by supervisors 
and trainees. 
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The continuance of Table 2 shows some of the data regarding th^ unique 
contribution of the overall assessment process * In both studies, the CAR 
adds some additional ability to account fur the criterion after the test 
data are controlled for* In most cases these are sizable increases. 

Operational use of predictions; control group * In the next group of 
studies, the assessment results were used in the company for promotion and 
development purposes, therefore the possibility of criterion contamination 
though knowledge of the assessment evaluations is a potential changer. 

The other feature of the designs of this group of studies is the com- 
parison of criterion dat- for a control group who were not assessed. 

An additional reservation with regard to this design is .the possibility 
that any positive regard for the asseasemnt center program among managers 
may lead to spuriously high ratings for assessed groups with little 
regard for actual performance. 

The first two studies are ones done at AT&T involving similar 
designs and criteria* Criterion data were obtained for groups of managers 
who had been promoted to a supervisory job. both a composite performance 
measure (Including the man's last formal appraisal rating and a rating 
and ranking by the supervisor) and a composite potential measure (including 
the rating of potential from the last formal appraisal, a potential ranking 
by the supervisor, and the present level of the man) were obtained. Some 
of the subjects had been assessed and some had not bben assessed. 

In the Bray & Campbell (1967) study, among the assessed group, the OAR 
ranged from "acceptable" to "not acceptable". The data for the perform- 
ance criteria show that there is a relationship between OAK and the 
criteria, but that a large portion (55-63%) of the non-sesessed managers 
who had been promoted were above average. The data on the potential 
criteria arc more clear-cut, showing higher success rate among assessed 
' • .. * v« CC\ . 
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than non-assessed groups. Both the performance and potential criteria in 
the 1964 Bray study shows predictive validity. 

The validation study of the assessemnt center at the Oak Ridge Gas- 
eous Diffusion Plant of the Union Carbide Corporation (Bender, Calvert, 
Jaffee, 1970; Jaffee, Bender and Cavert, 1970) includes some unique 
features. Due to the raall numbers of persona assessed at the time of 
the study, it was not possible to compare on-the-job performance of persons 
assessed at various levels of acceptability. Therefore, the design included 
a control group of 13 subjects promoted immediately prior to the initiation 
of the assessment program and an experimental group of 13 subjects were 
assessed. Analyses showed that the two groups were comparable on a number 
of important variables, but that the experimental group was younger, had 
less company service, and had been in supervision a shorter period* Criteria 
information were collected in. two areas (1) objective data of performanace 
of the subject’s work group, i.e. absences, grievances, or visits to the 
infirmary, (2) interview data from the supervisors and randomly selected 
subordinates regarding the adequacy of the subject as a supervisor. 

No differences in the objective criterion were found between the 
experimental and control groups. The study was done about one year aftev 
assessment and thus 'considering the short time in a supervisory capacity 
for most of the experimental group it is not at all surprising that these 
data did not reveal Any meaningful differences (Bender, et. al., 1970, 

P. 32)." 

The interview data showed that more positive comments ai-d fewer negative 
comment a were made about the performance of the experimental group than 
about the control group. This pattern held for both the superiors and the 
subordinates (except that the subordinates made move negative comments 
regarding assessed subjects). The authors warn that "These results n:ust 
certainly be considered carefully and no conclusions of a definite nature 
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may be drawn, but they are certainly indicative of a trend (Bender, et. 
al., 1970, p. 33)/' A major limitation of the study of course is the small 
sample size and the potential positive halo surrounding the assessed group. 
The studies we have reviewed up to this point have been generally 
supportive of the predictive validity of the assessment center techniques. 

In my critique of these studies we have noted the design flaws which tnay 
limit the usefulness of the results and their- generality. The only 
published study (Bullar, 1969) which is wholly negative involves the assess- 
ment center at Caterpillar Tractor Company. Bullard (1969) reports a 
study of (1) the relationship between assessment center predictions and 
two measures of future managerial behavior and (2) a comparison of the 
assessment center and a traditional method of selection and placement, 

Caterpillar's assessment center is very similar to the ones who* hirve 
reviewed thus far. Included in the assessment battery are a number of 
situational exercises and leaderless group discussions. Following the 
two-day evaluation period, the assessment staff spent the remainder of the 
week consolidating their observations rating the individual's performance 
on a 25-item Supervisory Qualification Rating Form (SQRF) and deriving 
an overall rating for each candidate: more than acceptable, acceptable, 

not ready now, and low in managerial skill. The SQRF is used at one of the 
criterion measures, being filled out subsequently by the general foreman 
on the Job. The SQRF includes 25 behavioral characteristics considered 
important managerial skills and traits, e*g. company attitude, importance 
of work, inner work standards, motivation, drive, etc. A brief description 
of each trait is given on the form followed by a five-point rating scale 
from low to high. No additional descriptora or qualifiers are given for 
the scales points. The form may suffer from lack of clarity and objectiv- 
ity. No data on its reliability are provided. The second criterion was 
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the Personal. Description Check List (PDCL), a list of 108 specific on-the- 
job behaviors, e.g. rarely takes chances, cooperates when requested, is 
too shy to be a leader, etc* This form was not used during the assess- 
ment center so no direct comparison can be made for this instrument* 
Criterion data were gathered for 37 experimental (assessed) sub- 
jects comprised of 4 groups from different plant locations and for 27 
control subjects from the same locations* The control subjects had been 
placed on their Job by the traditional method* Bullard (1969) states 
"Absolute control for age. time on supervision, and tenure with the 
Company was difficult in view of the small N (p5). M The report is not 
clear as to when the experimental and control supervisors were placed 
on their Jobs, If the conttol subjects were promoted prior :o the 
institution of the assessment center several problems could arise. First, 
the controls would have been on the job for a longer period at the time 
of criterion measurement; it would be likely that their job performance 
would be greater. Si cond, it may be that the experimental subjects were 
drawn from a very different pool of potential supervisors than the controls. 
The controls may be the "cream of the crop” selected at one point in time 
and thejx the experimental group was selected from a pool with generally 
lower abilities* This is pure speculation, for the Bullard (1969) re- 
port does not provide sufficient information. 

With the question of comparability of subjects in mind we can exa- 
mine the mean criterion scores for experimental and control groups in 
Table 2. On both the PD CL and the SDRF, the control group scored higher 
than the experimental f;roup« Analyses for the subgroups at the 4 locations 
follow the same pattern. Bullard (1968) interprets these results ,f as 
evidence that if superiority is to be aligned to either group on the 
basis of mean performance, then it must be assigned to those supervisors 
which constitute the control or "traditional method 1 group (p.l2-13). ,f 
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Operational use of predictions ; no control group . The last group of 



studies are those involving no control group and ones in which assessment 
data were used in the company. The data are generally supportinve, but the 
possibility of criterion contamination limit the generalizations from the 
fiudings • 

The studies of IBM (Hinrichs, 1969; Wollowick & McNamara, 1969) 
provide evidence of the predictive validity of assessment center Judgments 
and also show in what ways the situational exercieea contribute to the 
evaluations. Wollowick and McNamara (1969) report a study of 94 lower and 
middle managers who were followed up 3 years after assessment. The criterion 
was increase in managerial responsibility based on a complex index including 
number of persons supervised, complexity of Job, financial responsibility 
and 8**111 requirements of the Job. There were 12 increments in this index 
and the authors believe it is a better measure of advancement than simply 
level in the organization. Several results from the suudy are presented in 
Table 2 and its appends. The first correlation (^.37) indicates that the 
overall assessment rating of management potential is a fairly accur a e 
predictor of progress. But the best cccfolnatlon of psychometric tests 
correlated .45 with the criterion, suggesting that some predictive efficiency 
is lost when the judgmental process worka 1 on all the assessment data. This 
higher correlation with the multiple regtession equation may result from 
statistical artifacts in compitalizing or chance errors. All results should, 
of course, be crass -valid a ted. Of even more significance is the Jump from 
.45 to .62 when then the tests and exercises sod included in the multiple 
regression analysis. The percentage of criterion variance accounted for in- 
creases from 20% to 36%. 

Wollowick and McNamara (1969) put forth two tentative conclusions ba»ed 
on these results. First, the situational exercises contribute a significant 
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amount to the assessment of Important characteristics related to management 
success <R increases from *45 to .62). Second, the mechanical combination 
of separate sources of data gathered In the assessment center may improve 
predictive effectiveness over and above the subjectively determined overall 
rating. The first conclusion is similar to other results we cited in earlier 
sections; the second conclusion fits in with the general argument in favor of 
mechanical and actuarial prediction methods* No other study or assessment 
centers makes a comparison such as Wollowick and McNamara, but their study 
suggests strongly that some type of systematic weighting of inputs may 
Improve predictions of progress* 

Wollowick and McNamara warn that their results should be cross- val idated 
and this is always sound advice in multiple regression studies with a large 
number of variables and a small number of subjects* In lieu of actual 
cross-validation, it is possible to get some idea of the results vej might 
expect. Shrinkage" formulas, although not completely satisfactory (Horst, 

1966) give an estimate on cro^s-validation to a new sample (Blum 6 Naylor, 

1968)* When this type of analysis is done with the data in Wollowick and 
McNamara (1969) w e see only a slight reduction in multiple correlations 
and percentages of criterion variance accounted for. Another cause for 
placing some confidence on the results of this study is the generally acceptable 
reliabilities reported for the situational exercises (Greenwood 6 McNamara, 

1967) . Ratings and rankings of the leaderless group discussion, Msk-force 
cooniittee discussion, and the manufacturing game were shown to have inter- 
rater reliabilities predominantly in *70’s and *80*8* For one group of 
subjects the reliabilities were more typically in .60's* Of further note, 
were the comparisons between professionals and non-professionals* These 
correlations were somewhat lower (in a few cases near zero) but the N*s 
were small lu all cases* In summary r The results of this Study suggest 
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that reasonable rater reliability can be obtained in situational tests 
commonly used to assess . » • potential for advancement (p. 105, Greenwood 
& McNamara, 1967)." 

Hinrichs (1969) provides further data ou the IBM assessment programs 
in a study of 47 marketing recruits being considered for first level manage- 
ment. The study is unique in several regards and requires some close attentin 
For our purposes in this section, we can examine three variables in the study: 
(1) the overall evaluation of management potential resulting from deliberations 
of results of situational exercises and tests (internal criterion), (2) 
a measure of relative salary standing (external criterion), and (3) manager^* 
evaluation of potential (parallel criterion). Each of these criteria are 
examired more closely below. Of particular interest to us now are the data 
in Table 3, Based on these and other data, Hinrichs (1969) concludes: 

The data suggest that traditional approaches to the assessment 
of management potential in the form of careful evaluation of personnel 
records and employment history (our parallel criteria) can perhaps 
provide much the same information which evolves frcm the lengthy 
and expensive 2-day assessment programs , . . (p. 43). 

We suggest that the degree of overlap between the two assessment 
methods (which in fact is not highly substantial) is not adequate evidence 
of the effectiveness of either of them for assessing management potential. 
Hinrichs acknowledges that the data are not adequate for Judging validity 
and we agree. The author does tend to cast the overall assessment evaluation 
in terms of a "criterion 11 for comparison with other "criteria." This places 
high value on the assessment center approach.. We would prefer to view the 
assessment center as a predictor and question its value in relation to harder 
criteria. The use of the external criterion of relative salary i« not 
entirely adequate for this purpose. The purpose of the program was the pre- 
diction of management potential - a variable requiring longitudinal measurement. 
The salary criterion vss a measure of current value based on the relationship 
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of the individual's salary in comparison with that of his peers. 



Table 3 



Intercorrelations Among Criteria 



1 2 3 



r 



13-2 



Variable 



L. Relative salary standing (external criterion) 
2. Manager's potential evaluation (piaralled 



.10 .37 .37 



criterion) 



.46 



3. Assessment program evaluation (internal 

criterion) -- 

In the past five years, Wickes has used two assessment programs for 
retail lumber yard employees (Ague, 1971). The four-day program for assist- 
ant managers is primarily developmental and enables Wickes to determine 
when they are ready for promotion. In five years 184 assistant managers have 
been assessed and 126 have been promoted to manager. The annual turnover 
rate of assistant managers dropped form 307, prior to the program to 87> for 
assessed assistant managers. The Wickes Hourly Assessment Program has 
processed 240 hourly employees in the past five years. One hundred and twenty 
nine have been promoted to their management training program. The annual 
turnover for these 129 hourly employees, after initial management assignments 
is about 37. (Ague, 1971). No comparative data for the hourly turnover 
figure were presented. 

Turnover statistics reported by Wickes are a unique criterion which 
other companies might also use. These figures at Wickes represent very low 
turnover rates in the years after the program was Initiated. From the in- 
formation available, it is impossible to tell whether the reduction in turn- 
over is duo to the assessment program or to some other changes in the com- 
pany or in the economy as a whole. If the company launched a broad change 
In managerial philosophy or the company was experiencing rapid growth and 
financial success, there are any number of potentially valid explanations 
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of reduced turnover. Certainly more information is needed, but the results 
are very supportive and favorable for the impact of the assessment process. 

Although the assesement center approach has been used most frequently 
in industrial settings, it has also been applied in government organizations. 

A few of these include the Tennessee Valley Authority (Epenbach, 1971), 

Public Service Commission of Canada (Grant & Slivinski, 1970; Slivtnski 
& Grant, 1970), and the U.S. Internal Revenue Service (DiCostanzo & Andretta, 
1970'; Eeny, Meeny, miney , . . Supervisor?, 1970). Relatively little data 
exists on the validity of assessment centers in these types of settings. 

The IRS has carried out a small scale evaluation project (DiCostanzo & 
Andretta, 1970). Eleven offices were involved and 142 persons were assessed. 

In discussing the results in comparison with traditional evaluative infor- 
mation, the authors note that "there wes no correlation between relative 
strengths and weaknesses shown on (supervisory) evaluations and the correspond- 
ing areas in the assessment center reports . . • feedback from local manage- 
ment officials who used the reports indicated that In many cases they probed 
deeper with the supervisor and satisfied themselves that the Assessment 
Center findings were correct (p. 14, DiCostanzo & Andretta, 1970) 



The problems of obtaining good criterion data are apparent in the IRS 
situation ws we have observed in other companies. The problem of contamination 
is magnified by an orientation program run for managers. The IRS ran three 
two-day assessment center workshops, at which time over 75 higher level 
managers learned of the history and theory of assessment centers, the detail 
of a program, and actually went through a miniature assessment program. 

The important thing is that all these individuals were persons who would 
have access to and who would use the assessment center results (DiCostanzo 
6 Andretta, 1970). It is quite possible that a favorable set might be de- 
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veldped toward any supervisor who is recommended by the assessment process, 

tlue precluding the objective assessment of on-the-job performance. 

The TVA initiated a pilot program in an assessment center for managers 

in the divisions of engineering design and construction, water control, 

forestry, and navigation (Epenbach, 1971). The standard types of exercises 

were ustd and the evaluators (higher level managers) met to compile their 

ratings of performance. Independept Judgments were made by the evaluators on 

each candidate for 15 managerial skills. Erpenbach (1971) concludes: 

The MSAP (Management Skills Appraisal Program) was judged to have been 
highly successful by the experimenters. They found a high degree of 
relationship between the appraisers 1 ratings and the ratinga of the 
appraised employees 1 supervisors. Some at TVA, however, feel that the 
supervisors 1 ratings alone are adequate since they ace best qualified 
to assess their employees 1 potential. Because of budget limitations 
the pcograiu was discontinued, and there are no current plans to con- 
tinue it (. 2). 

Several interesting inferences can be drawn from the brief amount of 
information ara. table. 

The assessment programs of Standard Oil (Ohio) have been studied and 
reviewed quite thoroughly by their corporate staff (Finkle and Jones, 1970; 
Finley, 1970; Carleton, 1970) and outside researchers (Donaldson, 1969; 

Thomson, 1969, 1970). These studies have provided very thorough analyses of 
the internal and external validity of the assessment data. Of particular 
interest is the relationship between assessment ratingc and on-the-job 
ratings for the seme 13 behavioral traits. Three different sources (Carleton, 
1970; Finley, 1970; Thompson, 1970) provide data relevant to the predictive 
validity of assessment trait ratings for tnait rating of the job some years 
in the future. It can be concluded that assessors are able to validly 
predict behavior on several behavioral dimensions as a result of assessment 
center activities. Most of correlations are low to moderate, but the relation- 
ships of ratings of Potential are consistently high ( i.e. in the .65 range). 

^ ' . * C a 
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Tvo traits (Orientation to detail' and Relationship to Authority) account 
for 6 of the 7 non-significant correlations. This may be partially ex- 
plained by the hypothesized curvilinear relationship between these traits 
and success. Carleton (1970) states that for these traits 11 intermediate 
ratings were considered more favorable than either high or low. (p. 565)” 

The criteria in these studies could certainly be Subject to bias and 
contamination but the preas to distort the results is probalby minimal in 
comparison with the problems inherent in administrative criteria such as 
promotiona and salary data used so frequently in other studies we shall review 
below. The rating acales themselves used to gather the criterion ratings 
seem entirely adequate although we are not told how they were developed. 

Each of the rating instruments is labled, provides a definition of the trait, 
and includes a behavioral description of each of the five scale points (Thomp- 
son, 1969). 

Several questions might be aaked about these data. Since the correlations 
are only moderately high, it is possible that they are a result only of 
autocorrelation or method communal ity. Why aren*t they higher? Thompson 
(1970) has provided data relevant co theae questions and many more in a multi- 
t ra i t -mu 1 timet hod analysis of ratings. The matrix of correlations between 
13 traita and 3 sets raters (management assessors, psychologist assessors, 
and supervisors) were examined for convergent and discriminant validity. 

High levels of convergent validity (median r *■ .85) and discriminant validity 
were found for the two aaaessor groups on the 13 traits, but the supervisors 
ratings ©hewed lack of discriminant validity. The supervisors ratings shoved 
a general halo effect with little differentiation among the 13 tzaits. 

Thomson (1970) suggests that "the criterion raters were unable to arrive at 
a precise and common understanding of the meaning of the different scales, 
and responded to some generalized notion of the goodness or badness of the 
assessee (p. 501). 11 
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Summary and Conclusions . A review of the data in the second part of 



the paper suggest the following points: 

(1) The overwhelming general finding is that the GAR is related to a 
variety of criteria of subsequent performance and potential in 
managerial positions* 

(2) Serious questions regarding the quality of research designs and 
problems with criterion contamination in some studies limit the 
generality of the findings* 

(3) The overall assessment evaluation adds a unique contribution to 
assessment over and above the Information which can be obtained 
from objective tests. 

Final Surmary 

The published and private literature on assessment centers is generally 
supportive of the predictive validity of evaluations of management potential 
during the assessment process. Major desigr. flaws in some studies make con- 
clusions difficult. Of over 30 studies reviewed involving 21 companies, 
only one involved clearly no validity (Caterpillar) and one included strong 
reservations (Hinrichs, IBM)* Where comparisons are made with other methods 
of assessing management potential, assessment centers m&tce unique and 
substantial contributions. 

Virtually no objective data has been published or is available from 
sponsoring organizations which support the claim of the effectiveness of 
assessment centers as a management development technique* While admittedly 
it is difficult to measure some of the subtle outcomes that may be desired 
the burden of proof must fall on those who advocate the use of assessment 
centers for such purposes* 
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Table 1 



In* basket 



Valid tty of Individual Assessment Techniques 



Internal 

Validity 



External 

Validity 



Bray & Grant, 1966 



College 55 
Non college 51 



College groups: 27,-. 01, 

03,28 

Non college groups: 44, 

22, -19 



Wollowick 6 t McNamara, 1969 
Leader less Group Discussion 
Bray & Grant, 1966 



College 60 

Non college 38 



College groups: 30,50,26 

38 

Non college groups: 33, 

28,10 



Wollowick & McNamara, 1969 
Management Games 

Bray & Grant, 1966 



College 41 

Non college 42 



25 



College groups: 15,41,14 

-01 

Non college groups:, 37, 
50,29 



Hardesty & Jones, 1968 -05 

Wollowick & McNamara, 1969 .28 



Tests 



SCAT: 

Bray & Giant, 1966 
College 
Non college 



Wollowick & McNamara 
1969 

Bray & Campbell, 1968 

Hlnrlchs, 1969 

lUrdeaty & Jones, 

1960 



V36, Q06, T27 
v44, Q29, T41 



T03,03,-13 
V48, Q42, T53 



V36,35,51,30, 19,-44, 14 
Q23, 44, -04, 19,09,-10,-28 
T38, 45, 32, 28, 18,-30,-03 

Til 
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0 
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Carlton, 1970 



V29, Q36 
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Table 1 (eont.) 



Internal External 

Validity Validity 

(Note: For the following personality testa, data are number of 

scales with significant correlations. ) 



EPPS: 

Bray & Grant, 1966 2 

Gordon Tests: 

Hardesty & Jones, 0 

1968 



Hinrichs, 1969 

Wollowick, 1969 

Csrleton, 1970 

Study of Values: 
HInrichs, 1969 

Projective Techniques 

Grant, et. al., 1967 
College 
Non college 



Finley, 1970 
Sample I: 

Sample II: 

Interview 

Grant & Bray, 1969 
College 

Non college 



2 , 0,2 



0,0,0 



35 

40 



11 of 13 V*8 
significant 
13 of 13 v* s 
significant 



14 of 18 v f s 
significant 
8 of 18 v’s 
significant 



5, 3, 1,1, 1,0,0 



Ifl 



1,1 



For several projective 
variablea -01,10,11,-06, 
16, 24, -35, -25, 26, 12,19, 
21,-07,17,29,120,-23,30 



3 of 13 v‘s significant 
7 of 13 v*s significant 



9 of 18 v*s significant 
9 of 18 v's significant 
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Table 2 Validity of Overall Assessment Ratings (OAR) 






o 

ERIC 



® 

d 

o 

rt 

43 
<3 
rt 
TJ 

09 

& 

44 

o 

09 

V) 

3 



® 

Cl 

0 

rt 

4J 

03 

M 

01 
a 



o 

<y 

*§ 

w 





»o 










® 








5^ H 


rrt 




09 














O O 




O 


o 


u> 




q 








rrt C 




O so 






rrt 


• 




‘S 


rt 


<* 




CM 


09 JJ 




irt 










*0 


03 










•rt a 














o 


43 










44 














H 


4J 






















a 


03 


CM 


<t 




rrt 


30 09 














a 










m 


d rt 
















rt 










rt > 














n 


O 










4J 09 
















> 






09 U 














<* 


3 


1 

CO 


CM 






0 














M 












a 












0 


0 














OS CM 




o 


rrt 


9 


MM 
















<n 




rrt 


CM 


09 












ew 














4J 


CM 


























in 






a 




rt 














& 


• 






•rt 




O 














09 








cd 


















rt 


* 






4J 




• <s 














> 


rt 






4J 


• 


4J 






09 




U 




09 


m 








rrt 


0 






rt 




rrt 


09 


U 


« 






<y 


a 






33 




tO 


rrt 










•rt 


> 








® 09 




cO 


33 


TJ 


* 






•rt 


09 


rrt 




a 


4J »rt 


a 


4J 


03 


rrt 


r** 








rrt 


•rt 






0-33 




(X 4J 


09 


in 






> 




*rt 




x 


® c0 


i3 


09 


CL 


rt 


• 








T3 






4J 


O *J 


u 


o 


09 


CM 








•i 


rt 








d PX, 




u 


O 




* 






» <n 


ft 


oz 


09 


CJ 09 


® 




o 




aO 




a- 


a 




o 


lM 


o 


® 








<t 

« 




6 


i 




z 

1 


a 


£ 


(J 

< 






£ 





a a 



03 

0 

01 CO 
rt M 

£ 8 
o> rt 

a ® 






CO 

09 

8 

00 

2 

CM 

& 

JS 

* 

« 



h m 

09 vO 43 

> <ys co 
09 -H O 
rrt a. 

43 «5 09 



d ® 44 
rt M O 
03 O 
|M W 



a 



8*8 



C4 cs 
® ■ 



8 1 S S 

as *s 



ca 

w 



09 

O 

a 



3 «j oo <u 

43 I 09 
« 43 in 09 

X «d ^ « 



a 

® d *0 

M ® rt 

fl 49 rt _ 

•rt ft rt Cd 4-1 T3 

43 ® 44 V U U 

vs ‘m „ v a « 

K 4)^1 o a*o 



O tt 



§ a 

Is. 

* ® 

o> a 
00 o 
cr a 

rt 

rt <y 
O 00 
O 01 
HI 

CO rt X 
CM O *4 
rt O O 



3 

U 

O 

& 

« 

o 

rt 

0 

« 

00 

n 



gg 

°S2 

<« 

as 



a 

rt 

*d 

<u 

M4 

O 

V 

® 

d 



§ 

o 

rt 

4J 

03 

M 

09 

<& 



O 

a 

o <u 
> oc 

O CJ 
Ml 

a] 09 

. > 
S'* « 



44 

t4 

09 

a( 

09 Cl 

s « 

33 M 
to o 

. > 
^ 03 



O O rt 
in <* cn 



oo 

CM 



OO ID VO 
M3 vD <* 



•U 

O 

09 ® 
rt 03 
-Q d> 
CJ W 



SS 



CO 

00 

0 o; 

rt ex') 

*j a 

2 CO i-l ® 
00 43 00 

add 
09 rt « rt 

a -s - -a 

§£•2 <3 

o4e4 

44 Q9 
l< 43 
41 O 
PM PM 



*0 . 
<y 

CO M 
tt O 
a> ® 
w 
® 



cn 

vO 



® 

d 

o 



m 

m 



n 

g 



09 


33 


4J 


Ci 




43 


a 


43 


rrt 


d 


a 


< 


09 


O 


w 


o 


.n 


a 


G9 




W 


0 


09 


0 


cd 


o 


U 


43 




O 


® 


O 


4J 


rt 


o 


0 


< 


u 


w 


K 


a 


4J 


d 


X 


J 


Pm 


< 


CM 


09 








43 




1 




*} 


09 


4J 


P3 


W 




a 1 




o 


d 


o 


s 


O 




)* 




< 


<y 


S5 




,PM 




Pm 





r> ci 
8 



8: 



B 

I 

w 

I 

a 

8 



29 






k 

o 

44 



TJ 

<y 

§ 

•g 



cd 
■H 

ti 
© © 
k ‘ 



oo in 
vo co 



fl 

© 



© 

00 

If 

& XI 

01 fS 

h 

51 

<44 CJ 
k XI 
01 O 
Ck (V. 



a 

> » 

*H a 

XI k 

cd o 
00 to 
<U *H 

" fc 

k © 

© a 



o 'S 






a 


© a 


04 0 






XI 


44 








a 


T3 








9 


*0 © 










a © 
© © 








k 


© 


• 






Cl 


© © 


44 






a. 


XI © 


© 






X 


a © 


t 4 






© 




k x* 








i oo 


ai © 


co 




a 


e a 


XI © 


ta 


on 


>ri 


O *«4 


XI 








O T> 


o a 






© 


k 


x © 






© © 


w © 


• ^ 






U XI 


> to 


XI 






d o 


•H © 








© © 


xi k 
















© 43 


© © 






•P 


44 3 


o xj 






© 


44 © 


a a 






© 


•A 








© 


T3 r*4 


© a 




« 5 

© d Hr 
■p o 



i* 

s§ 

i g 



<M 




<0 

« 

© 

© 

» 

<\ 



U-N 

at 



k 

CJ 

X! 

00 



cn xi 

00 fl 



o 

■H 



d 

00 



r-t 

00 



IX 



_ *p 
^ © 
00 k 
O 

o 

to 

|x » 



s e 9 



o 
a 

■§ d G o 
o 

k • Pk • 

sae « w . 



k 

04 



co 
© 

XI 

(0 
s 
us 

m o 
5 * pO • 
3 d « 

*0 09 W| 



M 

O I 
■•4 I 44 
> k t 4 

5 8* 
JS 2 5 



0 jT d o 

n O 44 

1 CO 

© op © 4 J 

43 q k a 

X» «k o Cl 
XI M M 
0 *4 Q 
M > txi 



g 





•a 


r4 -O 






■p 




© 


- ^ 


© 




© 




© 


v0 © 


14 




© 




« 


© 


© 




© 


V 


© 


-P © 


00 


•o 


© 


© 


© r-f 


© © 


9 


■p 


© 


© 


© 4 « 


« © 


fl 


© 


9 






8 5 



_ a 

© a i 

in. is. § 

o cm a 



O, 

s 

o 

o 

o 

c 



XI 

o 

*H 

*p 

© 

M 

d 



I 



cn 



§1 

k 1 



oo 

o 

XI 





O 






CO 


CO 




a 


& 




o 


o 




k 


a 




44 


k 




© 


© 


vo 


d 




m <t 


*»4 


• t 


*-4 


fl 


^ /*\ 


Cl 

© 


S 


rH CM 


Q 


H 




00 

5 

XI 

J2 S 

k -A 

I 2 

_ U 

fs <y 

<t k 



to 

ft 

a 



© 

© 

© 



© 

« 

© 

44 

O 



2 

U 

Cl 

5 

I 



© a 

k x 

a o 

to «-4 

3 d. 



* 



$ 

at 




« 

1 



•o • 

niN |>N 

k at © at 

«H 0 H 

I:i 



'5 



s S 

at vi 

rs 



3 P 



s 



at 

to 

*0 «-< 

•M * 

P « 



H SB 

£ i! 



Ct 

tO 

at 



IP p 



r-t a 

SJi 



a 

>» 



30 



09 

U 

O 

09 



o 

& O 5 
H N OH 
H 



*o 
0 h 
s^pg 

w 



ERIC I 



o 



<0 

k v4 

o> • > 
60 w w 
<9 > 01 
C Ui 

3 P 

£ w - 



03 

41 

00 

H 

W 

o 



00 

H 

O 

00 



•H 




0 > 


> 


> • 




43 




U m 
01 0) 






a 


a ft 




to 


a 


P *g 




Oc 


w 


a 3 








4> 








o * 








5 u 




n 




a 




o 




€0 01 




CO 




Si n 




vl 




0 01 




► 




V4 44 
4J 44 




*4 

01 




<ti *rl 




a 


/-> 


H t3 




p 


CM 


rt 


w 


CM 


|4 01 


o 


• 


*— < 


0 01 


4J 


CO 






01 

H 




£ 


0 4-1 


14 


a <u 




o 


« 


M 0 




»2 


o 


a *j 

CO *J 





00 *rt 
00 4 

8 $ 

a> 

<j 2 

44 

° 8 

CO 

OU to 

ft 60 

S 3 

g 2 

ft u 

u 

Ji 

a 



to ft 
V 
00 

co _ 

< O 




* 

in 

<o 



X 

N 

cn 



■X 

oo 

in 



X 

3 



* 

in 

cn 



* 

NO 

CM 



* 

On 

cn 



■X 

a 



vt 

U 



St g 

a a 

h 3 

a « 



3 

M 



■X 

rv. 

CO 



•X 

vf 



■X 

co 

<n 



■x 

CM 

m 



-d* 



■X 

m 

cn 



■x 

co 



* 

m 

<* 



•x 

ON On 

cn cm 



■x 

3 



00 

tn 



■X 

cn 

cn 



•X 

o 

m 



* -x 

f< m 

cn m 



on cm 

r-4 <fr 



*x -x -x -x 

mmincMst 

cn <t co i-H vo 



cm 



X 

i-i 

cn 



x 

<$ 



XX X 

H ^ H 

CM CM O CM 



CM CO 

r— 1 CM 



o 

CM 



X 

ON 

CM 



X 

in 

VO 



NO 

CM 



X 

s0 

cn 



x 

m 

CM 



X 

CM 



X 

ON 



X 

NO 

CM 




X X 

O ^4 

-0 <M 



X 

CO 

*o 



NO 

CO 



X 


X 


X 


X 




X 


X X 




X 


X 




On 


o 


CM 


00 


o 


o 


m On 


f*. 


NO 


m 


cn 


CM 


CM 


CM 


CM 


o 


CM 


r4 CM 


H 


CM 


NO 


CM 






g 








1 >N 


41 








it 




*r4 

41 




r-t 




Sij 


•-4 

a 








ft 




9 




•H 




0 14 


o 








f-4 

ft 




a 

ft 




3 




<< O 


a 






ti 


A 




o 




« 




ss 








vf 


* 

41 




ft 

u 


5 






41 


o 






V 

(LI 


a 




u 


•O 


0 


0 


> 








S 


« 

u 




a 


«9 

41 


41 


o 

ft 


a 










u 

« 




44 


£ 


g 


u 

o 


>> 

J3 41 


•3 












O 




ft 


V 


a v4 


H 








H 




►n 


r“1 


2 




i •g 


3 










4J 


U 


3 


2 


•p 


•H 3 


tt 




4J 




« 

h 


O 

a 


ft 

iH 


o 

s 


I 


1 

f-4 


U *r4 


41 

*0 


41 


0 

01 

u 




& 


if 


1 


<s 


4 


$ 


& £ 


1 


£ 


£ 





31 



Table 2 con f t 



Bray & Grant, 1966 



Bray & Campbell, 1968 



Wollowick & McNamara, 



Hfnrlchs, 1969 



O 




Unique Contribution of OAR 



Correlation of OAR & salary 



progress 


•48 .57 .51 .52 


Partial correlation with 




ability test constant 


.32 .39 .29 .42 


Correlation of OAR & field 




ratings 


.51 


R of best combination of 




4 tests 


.33 


Correlation of CAR & 




criterion 


.37 


Paper & pencil tests 


.45 


Tests, characteristic, 




& exercises 


.62 


Correlation of OAR & 




salary 


.37 


with manager* s 




evaluation partialed 




out 


.37 
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